Fast Convergence in Evolutionary Equiilbrium Selection

Stochastic selection models provide sharp predictions about equilibrium selection when the noise level of the selection process is taken to zero. The difficulty is that, when the noise is extremely small, it can take an extremely long time for a large population to reach the stochastically stable equilibrium. An important exception arises when players interact locally in small close-knit groups; in this case convergence can be rapid for small noise and an arbitrarily large population. We show that a similar result holds when the population is fully mixed and there is no local interaction. Selection is sharp and convergence is fast for realistic noise levels and payoff values; moreover, the expected waiting times are comparable to those in local interaction models. 1. Stochastic stability and equilibrium selection Evolutionary models with random perturbations provide a useful framework for explaining how populations reach equilibrium from out-of-equilibrium conditions, and why some equilibria are more likely than others to be observed in the long run. Individuals in a large population interact with one another repeatedly to play a given game, and they update their strategies based on information about what others are doing. The updating rule is usually assumed to be a myopic best reply with a random component resulting from errors, unobserved utility shocks, or experiments. The long-run behavior of the resulting stochastic dynamical system can be analyzed using the theory of large deviations (Freidlin and Wentzell 1984). The key idea is to examine the limiting behavior of the process when the random component becomes vanishingly small. This typically leads to powerful selection results, that is, the limiting ergodic distribution tends to be concentrated on particular equilibria (often a unique equilibrium) that are stochastically stable (Foster and Young 1990, Kandori, Mailath and Rob 1993, Young 1993). A common criticism of this approach is that the waiting time required to get close to the long run distribution may be exceptionally large. The intuition is that in games with multiple strict equilibria, if the noise level is small and the system starts out near an equilibrium that is not stochastically stable, it will remain close to this equilibrium for a very long time. Indeed, the time it takes to escape the initial (“wrong”) equilibrium may grow exponentially in the population size when the noise is sufficiently small (Ellison 1993, Sandholm 2010). Nevertheless, this leaves open an important question: must the noise actually be close to zero in order to obtain sharp selection results? This assumption is needed to characterize the stochastically stable states theoretically, but it could be 1 Acknowledgements. We thank Sam Bowles and Bill Sandholm for helpful comments on an earlier draft. This research was sponsored by the Office of Naval Research, grant N00014-09-1-0751. 2 that at intermediate levels of noise the selection process displays a fairly strong bias towards the stochastically stable states. If this is so the speed of convergence could be quite rapid. A pioneering paper by Ellison (1993) shows that this is indeed the case when agents interact “locally" with small groups of neighbors. The paper deals with the specific case when agents are located at the nodes of a ring network and they are linked with agents within a specified distance. The form of interaction is a symmetric coordination game. Ellison shows that the waiting time to get close to the stochastically stable equilibrium, say all, is bounded independently of population size, and that its absolute magnitude may be very small. The reason that this set-up leads to fast selection is local reinforcement. When the noise is sufficiently small (but not taken to zero), any small close-knit group of interconnected agents will not take long to adopt the action . Once they have done this, they will continue to play with high probability thereafter even when people outside the group are playing the alternative action . Since this occurs in parallel across the entire network (independently of population size), it does not take long in expectation until almost all of the closeknit groups, and hence a large proportion of the population, have switched to . In fact this argument is quite general and applies to a variety of “local” network structures and stochastic learning rules, as shown in Young (1998, 2011). A separate line of work studies the conditions for equilibrium selection when agents (myopically) best respond to their available information. Sandholm (2001) considers the case when agents best respond to random samples of size , and shows that any -dominant equilibrium is eventually reached with high probability from almost any initial condition, for sufficiently large population size. In particular, when the sample size is two and interaction is given by a symmetric coordination game with a unique risk-dominant equilibrium, say ( ), this equilibrium will be reached from any initial condition in which a positive fraction of the population plays . The intuition for this result is that for almost any population action profile, the expected change in the proportion of players is strictly positive. The key idea is to work with the deterministic approximation of the process corresponding to an infinite population; in this setting the -equilibrium is an almost global attractor. When the population is large but finite, the process is stochastic but well-approximated by the deterministic dynamics, hence convergence occurs in bounded time with high probability. (A similar deterministic approximation is used to analyze the diffusion of innovations in random networks; see Lopez-Pintado (2006) and Jackson and Yariv (2007).) However, none of these papers characterizes the expected waiting time to reach equilibrium. The contribution of this paper is to provide an in-depth analysis of expected waiting times in evolutionary models with global interaction and best response dynamics. The model assumes a large, finite population of identical agents repeatedly interacting according to a symmetric coordination game. Agents occasionally have the opportunity to revise their actions, and when they do they choose a perturbed best response to the distribution of actions in the population. We shall consider the case where agents know the actual distribution of actions and also the case where they only observe a finite random sample of actions. In the interest of analytical tractability we shall restrict attention to coordination games and logit best response dynamics (Blume 1993, 1995). For the sake of concreteness we shall call action the innovation and action the status quo and 2 An equilibrium ( ) is -dominant if it is a best response to choose given any sample of size that contains at least one player playing . 3 talk about the adoption rate of the innovation starting from the status quo when everyone is playing . This setting allows us to characterize waiting times in terms of two easily interpretable parameters: the payoff gain of the innovation relative to the status quo, and the error rate . Our analysis employs a variant of the deterministic approximation technique used by Sandholm (2001). In our case, however, we shall fix the noise level at a level bounded away from zero. Some of the main results are the following: i. The dynamics exhibit a phase transition in the payoff gain. For any given level of noise there is a critical payoff gain such that selection is “fast”, that is, the expected waiting time until a majority play is bounded independently of population size whenever is larger than the critical value. Intuitively, for low payoff gains the deterministic approximation has three equilibria corresponding to fixed points of the logit function, whereas for sufficiently large payoff gains only the “high” equilibrium survives. ii. We provide a sharp estimate for the critical payoff gain and show that for moderate noise levels it is equal to zero. In other words, for these noise levels selection is fast for all positive values of the payoff gain no matter how small. iii. We provide an upper bound on the expected time as a function of the payoff gain . Simulations show that the estimate is accurate over a wide range of parameter values. iv. For realistic parameter values the absolute magnitudes of the waiting times are very small and quite close to those found in local interaction models (Ellison 1993). For example, we find that when the innovation results in an improvement over the status quo, the agents’ error rate is and they respond to the true distribution of actions in the population, it takes revisions per capita in expectation to reach an adoption rate of . When agents respond to random samples of size , this number drops to . The paper unfolds as follows. We begin with a review of related literature in section 2. Section 3 sets up the model, and section 4 contains the first main result, namely the existence and estimation of a critical payoff gain when agents have perfect information. We derive an upper bound on the number of steps to get close to equilibrium in section 5. Section 6 extends the results in the previous two sections to the imperfect information case, and section 7 concludes. 2. Related literature The rate at which a coordination equilibrium becomes established in a large population (or whether it becomes established at all) has been studied from a variety of perspectives. To understand the connections with the present paper we shall divide the literature into several parts, depending on whether interaction is assumed to be global or local, and on whether the selection dynamics are deterministic (best response) or stochastic (noisy best response). In the latter case we shall also distinguish between those models in which the stochastic perturbations are taken to zero (low noise dynamics), and those in which the perturbations are maintained at a positive level (noisy dynamics). To fix ideas, let us consider the situation where agents interact in pairs and play a fixed symmetric pure coordination game of the following form: